Typical computer systems, especially computer systems using graphical user interfaces (GUIs), are optimized for accepting user input from one or more discrete input devices, such as a keyboard for entering text, and a pointing device, such as a mouse with one or more buttons, for operating the user interface. An example of such a GUI is the user interface for the Windows™ computer operating system (available from Microsoft Corporation of Redmond, Wash.). The ubiquitous keyboard and mouse interface provides for fast creation and modification of documents, spreadsheets, database fields, drawings, photos, and the like.
Recently, however, pen-based computing systems, such as tablet PCs, personal digital assistants, and the like, have been increasing in popularity. In pen-based computing systems, user input advantageously may be introduced using an electronic “pen” or stylus (e.g., akin to writing with a conventional pen or pencil on a piece of paper). Indeed, in at least some pen-based computing systems, all user input is capable of being entered and manipulated using a pen, and the user interface is fully controllable using only the pen.
In at least some pen-based computing systems and the like, the pen input device is called upon to perform multiple different tasks, such as entering electronic ink, tapping icons or other objects (akin to a conventional left mouse button click), double tapping icons or other objects, initiating press-and-hold actions (akin to a conventional right mouse button click), moving icons or other objects, targeting on icons or other objects, resizing objects, and the like. These tasks may be accomplished by performing various different “gestures” using the pen. In order to avoid having the user enter data to tell the computer what type of input he/she is intending to enter before entering it, pen-based computing systems have developed algorithms that allow the system to “automatically” determine the intended type of input before, during, and/or after the input is being entered. In many situations, however, improving the pen-based computing system's ability to correctly recognize one or more types of system gestures will degrade its ability to quickly or correctly recognize other pen operations or gestures, sometimes even very basic pen operations or gestures.
For example, when attempting to recognize a pen “tap” gesture (akin to a mouse click action), one of the main factors evaluated by at least some pen-based computing systems relates to the amount of pen movement during the pen-down gesture that potentially constitutes the tap (e.g., the system has a threshold parameter indicating the amount of distance the pen is allowed to “skid” or move during a gesture while still having the system recognize the gesture as a tap). Often, particularly when people casually tap the pen down on the digitizer surface, the pen will skid or move a substantial amount during those few milliseconds while it is in contact with the digitizer. If the computing system were to simply start generating “mouse move” messages corresponding exactly to every pen movement action during a pen-down event (i.e., if the parameters for recognizing a “tap” gesture did not allow any pen movement in the X or Y coordinate directions during a tap), an icon (or other object) in a menu or other display region may be made to move (e.g., by activating a drag operation) instead getting clicked, such that the icon (or other object) would change position rather than getting selected. Such strict input threshold parameters would make it difficult to perform a tap gesture, which would be very frustrating to users.
Therefore, conventional pen-based computing systems include threshold parameters and tolerances that attempt to produce correct input gesture recognition despite the fact that the user's input may have been somewhat “sloppy” or less than ideal. For example, in commercially available pen-based computing systems that operate Microsoft™ Windows XP Tablet PC Edition Version 2002, when a “pen-down” event occurs, the system waits and does not generate “mouse move” events for a short time period after the pen comes in contact with the digitizer surface. After a few milliseconds, the system will be more likely to correctly determine whether the pen is going to continue traveling (e.g., consistently in one direction more than a predetermined threshold amount if the user is entering ink or dragging an object), or whether the pen will quickly stop moving or not move (e.g., thereby representing a “tap” gesture). As soon as the correct gesture is recognized (or believed to be recognized), e.g., based on one or more predetermined or preset threshold parameters, the previously delayed mouse move events may be sent, if necessary, and the processing may continue based on this gesture recognition.
Accordingly, pen input and gesture recognition in at least some existing pen-based computing systems evaluate various input characteristics during a pen-down event, and by balancing the different input characteristics against one another, the type of input is recognized (either correctly or incorrectly), and this input then is acted on by the system. If an operating system wanted to allow users to be able to input very sloppy taps (e.g., with substantial pen movement or skid) and still recognize that input gesture as a tap, then the operating system also would have to wait longer to make a decision as to whether a particular input gesture constitutes a “tap” gesture or a “drag” gesture or an ink input gesture, because the pen may move in the same general way during a sloppy tap vs. during ink entry and/or during a drag operation. The sloppier the tap input gesture that is allowed, the longer the recognition delay time period needs to be. On the other hand, this extended delay results in a perceived latency when the user does not intend to make a tap gesture. More specifically, when users intend to enter electronic input ink or perform a dragging operation, the user may feel as if the system is freezing briefly, almost as if the system is not capable of keeping up with the user input, because the system delays to determine whether a sloppy tap was input rather than the ink or drag input. Therefore, this “balancing act” can degrade the quality of drawing and dragging operations and the user's experience with pen-based computing systems.
The “tap” vs. ink entry and/or dragging operations are not the only gesture operations that may cause recognition delays in pen-based computing systems. For example: (a) improving the pen-based computing system's ability to correctly recognize double taps degrades its ability to correctly recognize consecutive single taps on closely spaced icons or objects (and vice versa); (b) improving the pen-based computing system's ability to correctly recognize “press-and-hold” gestures degrades its ability to correctly recognize ink entry and/or dragging operations (and vice versa); and (c) improving the pen-based computing system's ability to improve targeting activities degrades its ability to recognize ink entry and/or dragging operations (and vice versa). While the Microsoft™ Windows XP Tablet PC Edition Version 2002 made significant efforts to strike the right balances and provide suitable threshold parameters for recognizing these different types of input and gestures, improved gesture recognition ability and/or improved speed of correct gesture recognition would further enhance the pen-based computing system user's experience and would be a welcome addition to an operating system.
Pen-based computing systems also have “calibration” issues, issues that are not experienced with at least some other types of computing systems. For example, due to mechanical limitations relating to pens, displays, and digitizers, the digitizer hardware typically reports interaction pen positions (e.g., X,Y coordinates) that do not exactly match the location on the display where the visual target is located (e.g., this may occur because the display device that generates the visual target is different and independent from the digitizer device used to recognize the location of pen during input interaction). Other sources of calibration issues may include, for example: electromagnetic noise caused by hard drives and/or power supplies, sensitivity characteristics of the digitizer hardware (e.g., the digitizer's sensitivity may change with temperature, elevation, humidity, etc., such that its behavior does not stabilize until after a “break-in” period, and correct, perfect calibration is not possible from the factory due to this “break-in” period, etc.), and the like. To correctly interact, the user must touch the pen down at the digitizer location where the digitizer believes the icon or object to be located, irrespective of where the icon or object actually exists on the display screen. If the difference between the visual position of the target on the screen and the required digitizer interaction position on the digitizer is small, the user typically will not notice any problem and everything will work in the manner it should. However, in practice, this difference can be quite significant. As a result, a user may be physically pointing to and interacting with one location on the display screen, but he/she actually may be interacting with an offset location in the system as recognized by the underlying digitizer. This difference especially may manifest itself during tasks of “fine targeting,” such as when a user attempts to resize an input panel by dragging the panel's edge, when a user attempts to click on a small icon, button, or other object, such as the maximize, restore, or close buttons on an input panel, or the like.
Typically, these differences may be addressed using one or more “calibration” techniques, during which the pen position reported by the digitizer hardware is adjusted based on expected user input and interaction with a visual display during a calibration procedure. Examples of calibration procedures used for digitizers and/or other hardware involved in pen-based computing systems include “linearization” calibration, “four-point” procedure calibration, and the like.
Pen-based computing system manufacturers also may perform calibration procedures on pen-based computing system hardware before it is shipped for sale. For example, some manufacturers may fully calibrate a given machine coming off the assembly line (e.g., using a 256 point calibration procedure), and then they may use this same calibration data on the next predetermined number of machines coming off the line (e.g., perhaps the next thousand machines or more). While the limitations in extending the calibration data from one machine to other, independent machines are readily apparent, it would cost too much in time delays to perform this complex calibration procedure on every machine coming off the line. Other hardware manufacturers may do little or no actual calibration and rely on the likelihood that their manufacturing procedures have closely aligned the digitizer and the LCD screen and/or that the underlying circuitry of the computing system will not generate sufficient noise to substantially disturb the actual user experience. The limitations inherent in these calibration procedures (or lack thereof) are readily apparent, and these procedures do nothing to correct for calibration drift that occurs over time and/or system noise generated by electronic components making up the system.
While many pen-based computing systems allow users to enter a “calibration mode” or program, such dedicated modes rely upon the user's knowledge of the need for calibration and knowledge of the availability of such a mode or program. Moreover, the calibration activities and effectiveness available in such modes can be relatively limited. Even when conducted, these user-initiated calibration procedures typically are time-consuming, inconvenient, and temporary fixes. Accordingly, improved calibration procedures would be a welcomed addition to pen-based computing systems.